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Donald  A.  Topmiller 


INTRODUCTION  AND  HISTORICAL  ANTECEDENTS 

The  field  of  Human  Factors  Engineering  in  the  United  States 
evolved  out  of  demands  from  the  increasing  complexity  of  military 
weapon  systems  in  World  War  II.  The  early  workers  in  the  field  such 
as  Paul  Fitts  in  the  Air  Force,  Adelbert  Ford  in  the  Navy  and  a  lit¬ 
tle  later  John  Weisz  in  the  Army,  were  all  trained  as  experimental 
psychologists,  hence  the  methods  applied  during  these  years  were 
transitioned  almost  directly  out  of  empirical  applied  psychology. 

This  was  formally  recognized  in  1949  with  one  of  the  first  published 
texts  in  the  field  (Chapanis  et  al.,  1949). 

During  the  early  and  mid  1950s,  the  application  of  the  tools  of 
the  experimental  psychologist  to  man-machine  design  problems  continued 
to  flourish.  We  also  saw,  during  this  period,  an  attempt  to  extra, 
from  the  existing  experimental  literature,  human  performance  limits, 
given  certain  man-machine  interface  characteristics,  environmental 
conditions,  and  task  environments.  An  excellent  early  effort  to  do 
this  was  reflected  in  the  publishing  of  the  "Tufts  Handbook"  or  the 
Handbook  of  Human  Engineering  Data  in  1951.  This  was  the  authoritive 
reference  document  for  Human  Engineering  Data  for  over  ten  years.  In 
the  early  sixties  the  three  mil  itary  services  convened  the  Joint  Armv- 
Navy-Air  Force  Steering  Committee  to  publish  the  Human  Engineering 
Guide  to  Equipment  Design.  Th'e  first  edition  was  published  in  1963 
and  the  revised  second  edition  in  1972.  The  respective  services  also 
published  numerous  engineering  handbooks  of  human  engineering  design 
principles  and  criteria. 

The  underlying  assumption  during  the  "Handbook  Era"  was  that  de¬ 
sign  engineers  and  system  engineers  would  apply  the  appropriate  human 
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engineering  data  to  manned  systems  design  problems.  Frequently,  this 
was  found  not  to  be  the  case  for  several  reasons.  First,  many  design 
engineers  were  not  aware  of  the  existence  of  the  handbook  data,  or 
if  they  were,  there  was  very  little  incentive  to  use  the  data  in 
their  design  trade-off  analyses.  (This  was  often  the  fault  of  the 
government  program  management  since  no  requirement  for  the  applica¬ 
tion  of  human  factors  was  included  in  the  contract.)  Secondly,  if 
the  need  was  recognized,  the  design  engineer  could  not  apply  the  da¬ 
ta  since  the  problem  or  intended  operational  use  of  the  system  did 
not  correspond  to  the  conditions  under  which  the  data  were  experi¬ 
mentally  derived.  Finally,  an  integrating  structure  for  the  data 
base  which  was  systems  and  problem  specific  did  not  exist.  It  was 
frequently  found  that  in  order  to  solve  a  specific  design  problem, 
a  physical  simulation,  or  at  least  a  functional  mock-up,  of  the  manned 
system  design  was  required.  Depending  on  the  significance  of  the 
design  issue  and  the  expense  involved,  the  solution  may  or  may  not 
have  been  assured  by  physical  simulators. 

This  situation  was  probably  characteristic  of  the  human  factors 
state-of-the-art  in  the  mid  1960s.  The  experimentalist,  qua  human 
engineer,  found  that  experimental  methodology  used  in  problem  speci¬ 
fic  cases  was  often  extremely  expensive  and  may  or  may  not  address 
the  operational  or  system  variables  critical  to  the  design  issue.  We 
also  saw  during  the  same  period  an  explosive  growth  in  digital  com¬ 
puter  technology.  This  technological  development  was  exploited  by 
the  operations  research  community  and  led  to  the  rapid  computerization 
of  large  multivariate  systems  simulations  as  well  as  the  coincident 
development  of  computer  simulation  languages.  The  first  psychologist 
to  recognize,  and  eventually  implement,  the  power  of  the  computer  si¬ 
mulation  to  man-machine  system  design  problems  was  Arthur  Siegel  from 
Applied  Psychological  Services,  who  along  with  day  Wolf,  a  mathemati¬ 
cian,  under  the  Office  of  Naval  Research  contract  developed  the  first 
2-operator  model  and  in  1969  published  the  first  book  in  man-machine 
simulation  models  (Siegel  and  Wolf,  1969).  Certainly  this  development 
contributed  a  quantum  jump  in  human  factors  methodology  and  eventual¬ 
ly  led  to  several  man-machine  systems  design  and  analysis  techniques 
such  as  CAFF.S  (Computer  Aided  Function-Allocation  F.valuation  System) 
in  the  Navy,  with  its  associated  submodels,  and  SAINT  (Systems  Ana¬ 
lysis  Integrated  Network  of  Tasks)  in  the  Air  Force.  Human  factors 
engineering  recognized  that  in  order  to  get  man-machine  design  prin¬ 
ciples  integrated  into  system  design  and  development  programs,  it 
would  have  to  borrow  heavily  from  the  systems  simulation  and  opera¬ 
tions  research  communities. 

In  fact,  in  the  mid-sixties  our  Laboratory  sponsored  a  contrac¬ 
tual  survey  with  Air  Research  Incorporated  to  evaluate  the  extent  to 
which  human  factors  (man-machine  interface)  parameters  had  been  in¬ 
corporated  into  military  Operations  Research  (OR)  studies  (Schwartz 
et  al.,  1967).  A  total  of  250  studies  were  reviewed  covering  a  period 
from  World  War  II  until  1965.  Detailed  analysis  was  made  on  20  of 
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the  most  representative  studies  to  evaluate  the  extent  to  which  hu¬ 
man  factors  variables  were  incorporated  into  the  study,  and  the  de¬ 
gree  to  which  the  model  was  sensitive  to  variations  in  the  human  fac¬ 
tor  parameter  values.  The  detail  review  included: 

-  Description  of  the  system  modeled. 

-  Discussion  of  the  problem  treated  and  objectives  of  the  analysis. 

-  Identification  of  the  OR  techniques  used.  (Hopefully  a  "classical" 
OR  model . ) 

-  Identification  of  the  human  factors  parameters  relevant  to  the  mo¬ 
del  of  the  systems. 

-  Discussion  of  the  role  and  treatment  of  human  factors  parameters 
found  in  the  model. 

-  Discussion  of  the  sensitivity  of  the  model  to  changes  in  values  of 

the  human  factors  parameters. 

From  the  analysis  ensuing  from  the  above  objectives,  the  study 
lists  the  following  conclusions: 

-  There  is  a  strong  tendency  for  OR  investigators  to  concentrate  too 
heavily  on  the  model  rather  than  the  natural  system  problems. 

-  The  relative  sensitivity  of  various  OR  techniques  to  variations  in 
human  factor  variables  were  difficult  to  assess  for  the  models  sur¬ 
veyed  because: 

a.  The  majority  of  OR  analyses  do  not  analytically  treat  human  per¬ 
formance  parameters  (affecting  system  performance),  since  they 
seem  imbedded  in  the  terms  of  various  OR  models,  but  precise  re¬ 
lations  among  the  parameters  and  OR  model  terms  are  not  demon¬ 
strated  . 

b.  In  the  studies  which  treat  human  performance  variables,  the  pa¬ 
rameters  considered  and  systems,  or  contexts,  modeled  are  too 
diverse  to  permit  direct  and  quantitative  comparison. 

c.  The  lack  of  standard  measure  of  human  performance  in  systems 
serves  to  increase  the  complexity  of  the  problem. 

In  retrospect  this  is  a  lesson  in  irony  since  Siegel  and  Wolf 
at  that  time  had  not  received  wide  publicity  in  their  model  develop¬ 
ment  efforts,  but  must  have  recognized  these  deficiencies  which  stim¬ 
ulated  their  use  of  monte-carle  techniques  for  developing  the  orig¬ 
inal  2-man  model 

At  the  turn  of  the  decade  into  the  seventies,  we  saw  these  ma¬ 
jor  trends  in  man-machine  design  methods  and  techniques  development. 
First,  with  the  stage  set  by  the  technological  breakthrough  in  com¬ 
puter  power  many  human  factors  groups  in  industry  and  government  ei¬ 
ther  developed  their  own  computer-controlled  engineering  design  simu¬ 
lation  capability  for  performing  single-operator  and  multi-operator 
(crew)  real-time  mission-based  simulation  or  they  piggy-backed  stud¬ 
ies  on  simulation  facilities  developed  and  controlled  by  hardware 
engineering  groups.  Many  of  these  facilities  were  used  to  investigate 
advanced  cockpit  designs,  advanced  control-display  concepts  and/or 
measurement  of  pilot/crew  workload.  It  was  fortuitous  that  this  high- 
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technology  capability  in  man-machine  design  physical  engineering  simu 
lation  filled  a  unique  requirement,  since  during  the  seventies,  the 
national  defense  policy  was  one  of  enhancing  existing  weapon  system 
capability  by  hardware/software  modification  and  upgrade  rather  than 
investing  in  new  weapons  per  se.  This  was  particularly  true  of  the  de 
cisions  not  to  produce  the  B-l  and  instead  upgrade  the  avionics  (elec 
tronic  warfare  and  navigation)  capability  of  the  20  year  old  B-52. 

Our  Laboratory  developed  the  SACDEF  (Strategic  Air  Command  Design 
Evaluation  Facility)  to  evaluate  the  operator/crew  performance  as¬ 
pects  of  the  "improved"  avionics  systems.  Our  Laboratory  also  de¬ 
veloped  a  computer-based  multi-operator  command/control  simulation 
facility  on  which  many  simulation  of  BUIC  (Back-Up  Interceptor  Con¬ 
trol)  ,  AWACS  (Advanced  Warning  And  Control  System)  and  RPV  (Remotely 
Piloted  Vehicle)  surveillance  and  weapons  direction  mission  were  con¬ 
ducted.  This  program  is  summarized  in  a  chapter  by  the  author  in 
Tsokos  and  Thrall  (1979).  In  fact,  in  our  BUIC  simulations  we  at¬ 
tempted  for  the  first  time  to  integrate  computer-simulation  with 
multi-operator/multi-task  real-time  physical  simulation.  Where  we  had 
the  capability  to  conduct  the  physical  simulation  of  the  operational 
BUIC  "active  tracking"  tasks,  we  did  so.  Those  operational  tasks 
which  could  not  be  physically  simulated,  by  virtue  of  computer  and 
display  limitations,  we  computer-simulated  using  a  Siegel-Wolf  model. 
This  enables  us  to  combine  the  respective  powers  and  advantages  of 
both  simulation  disciplines  into  one  "hybrid"  technique.  It  was  in¬ 
deed  interesting  and  gratifying  to  find  that  the  independent  vari¬ 
ables  we  were  manipulating  in  the  experimental  simulation  such  as 
radar  track  trail  length  and  penetrator  velocity  had  significant  ef¬ 
fects  on  task  performance  measures  such  as  "tracker  initiation  time" 
and  were  not  washed-out  by  the  potential  propagation  of  sampling  er¬ 
ror  in  the  monte-carlo  process  in  the  Siegel-Wolf  model.  Unfortunate¬ 
ly,  for  a  variety  of  reasons,  we  have  not  followed  through  on  this 
"hybrid"  technique  to  develop  it  to  its  highest  potential  as  a  manned 
systems  simulation  and  design  tool. 

In  the  thirty  some  odd  years  covered  in  this  very  brief  and 
sketchy  treatment  of  the  historical  antecedents  to  the  methods  used 
in  manned  systems  design,  we  have  seen  our  field  evolve  out  of  the 
early  work  in  applied  experimental  psychology  through  the  design 
handbook  era  and  becoming  more  interdisciplinary  with  strong  techno¬ 
logical  influences  stemming  from  the  fields  of  computer  and  informa¬ 
tion  science  with  overtones  of  operations  research  and  systems  simu¬ 
lation  and  modeling. 


CURRENT  TRENDS 

The  US  Air  Force  Systems  Command  has  for  the  past  year  and  a 
half  conducted  a  very  comprehensive  study  of  the  Human  Factors  Engi¬ 
neering  (HFE)  field  and  its  technology  base  development,  application 
of  principles,  tools,  methods,  and  design  criteria  throughout  the  re- 
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search  and  development  phases  of  AF  Weapons  Systems  Acquisition.  It 
also  included  extensive  examination  of  HFE  problems  associated  with 
acquiring  and  developing  professionals  in  the  Air  Force,  both  civil¬ 
ian  and  military  to  meet  the  demands  of  advanced  weapon  system  de¬ 
velopment,  acquisition  and  operation. 

The  study  was  conducted  primarily  by  a  select  group  of  senior 
nationally  and  internationally  renowned  human  factors  engineers  who 
performed  under  contract  to  the  Air  Force.  Certain  senior  military 
and  civilian  HFE  worked  with  the  contract  professionals  to  provide 
needed  governmental  information. 

Robert  C.  Williges  and  the  present  author  worked  collaborative- 
ly  on  one  of  the  four  task  committees  to  perform  the  "Technology  As¬ 
sessment"  (Williges  and  Topmiller,  in  press)  task  the  results  of 
which  were  relied  on  heavily  to  develop  this  section  on  Current 
Trends  and  the  final  section  on  Future  Requirements. 

For  purposes  of  the  "Technology  Assessment"  task,  technology  in¬ 
cludes  information,  methods,  and  concepts/devices  created  for  useful 
Human  Factors  Engineering  (HFE)  purposes.  These  technologies  were 
classified  into  the  following  categories: 

1.0.  Reference  Data  Sources  -  These  are  catalogs  of  already  collected 
data  which  maybe  measures  of  human  properties,  limits,  tolerances 
task  performance  capability,  interface  design  principles,  design 
criteria  and  operator-machine  centered  equipment,  subsystem  or 
system  data. 

2.0.  Experimental  Design  Methods  -  are  means  by  which  data  are  col¬ 
lected  to  enhance  the  data  bases,  to  draw  inferences  and  to 
solve  problems. 

3.0.  Human-Machine  Integration  Performance  Metrics  -  are  measurements 
used  in  data  collection/problem  solutions. 

4.0.  Models  -  are  means  of  organizing  information  to  represent  the 

functioning  of  objects/processes  being  modeled  by  imposing  for¬ 
malistic  rules  and  relationships. 

5.0.  Engineering  Design  Simulation  -  is  concerned  with  the  use  of 

simulation  as  a  design  tool  and  is  not  to  be  confused  with  the 
production  or  use  of  training  simulation. 

6.0.  Procedures  -  are  rationally  organized  steps  to  aid  in  the  pro¬ 
duction  of  a  design  (at  any  stage),  evaluation  of  a  design,  or 
in  extraction,  extension  or  analysis  of  a  design. 


To  idenfitv  specific  HFE  technologies  falling  under  the  six 
broad  categories  a  total  of  one  hundred  and  thirty-six  (136)  subcat¬ 
egories  of  technologies  were  used  for  collecting  the  data  base  from 
which  the  study  developed  its  findings  and  conclusions.  Table  1  lists 
the  aggregate  of  technologies  examined. 
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Table  I.  Listing  of  Current  HFE  Technologies 


1.0.  Reference  Data  Sources 

Scientific  &  Engineering  Data  Sources 
Professional  Journal  Literature 
Handbooks,  Guides,  Specs 
Design  Handbook  1-3 
NASA  Bioastronautics  Data  Book 

Design  Handbook  for  Image  Interpretation  Equipment 
Human  Engineering  Guide  to  Equipment  Design 
MIL  Spec  1472 
Data  Bases 

Anthropometric  Source  Book  &  Data  Bank  (static  &  dynamic) 

Old  Systems  Record  Review 

SHERB  (Sandia  Human  Error  Rate  Bank) 

HFTEMAN  (Human  Factors  Test  &  Evaluation  Manual) 

2.0.  Experimental  Design  Methods 

2.1.  Statistical  Procedures 

Univariate  Methods 
Correlations 
Simple  Regression 
Parametric  Inferences 
Nonparametric  Inferences 
Multivariate  Methods 
Multiple  Regression 
Polynomial  Regression 
Canonical  Analysis 
Principal  Components 
Factor  Analysis 

MANOVA  (Multivariate  ANalysis  Of  VAriance) 

Discriminate  Analysis 
Pretesting  Methods 

ANOVA  Designs  (ANalysis  Of  VAriance) 

DATA  Reduction  Designs 
Blocking  Designs 
Hierarchical  Designs 
Fractional-factorial  Designs 
Central-composite  Designs 

2.2.  Tailored  Methods 

Confusion  Matrices 
Quasi-experimental  Designs 
Response  Surface  Methodology 
Finite  Interaction  Test 

3.0.  Human-Machine  Integration  Performance  Metrics 

3.1.  Anthropometric  Measures 
Static 
Dynamic 
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Table  I.  Listing  of  Current  HFE  Technologies  (Continued) 


3.2.  Physiological  Measures 

FFF  (Flicker  Fusion  Frequency) 

GSR  (Galvanic  Skin  Response) 

EKG  (Electrocardiogram) 

EEG  (Electroencephalogram) 

ECP  (Evoked  Cortical  Potential) 

Eye  and  Eyelid  Movement 
Pupillary  Dilation 
Muscle  Tension 
Heart  Rate 
Breathing  Analysis 

3.3.  Human-Machine-Environment  Measurement  Techniques 

Noise  Map/Fill 

Vibration 

Impact 

Acceleration 
Noise  Mapping 

3.4.  Subjective  Opinions 

Questionna ires /Check lists/ Ratings 
Open-Ended 
Multiple  Choice 
Rating  Scales 
Ranking  Procedures 
Forced  Choice 
Semantic  Differential 
Critical  Incidents 

3.5.  Automatic  Recording  Methods 

Event  Recording 
Photography 
Audio/Video  Tapes 
Motion  Pictures 

OPRENDS  (Operational  Performance  Recording  and  Evalua¬ 
tion  Data  System) 

Recorded  Flight  Data 


4.0.  Models 

4.1.  Biomechanical  Models 

Architectural  Body  Models 
Dynamic  Dan 
Combi man 

4.2.  Performance  Models 

Information  Theory 
Statistical  Decision  Theory 

TSD  (Theory  of  Signal  Detectability) 
Bayesian  Decision  Making 
Estimation  theory 
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Table  I.  Listing  of  Current  HFE  Technologie  (continued) 


Control  Theory 

Quasi  linear  Control 
FFM  (Fixed-Form  Models) 

OCM  (Optimal  Control  Models) 

Queueing  Theory 
4.3.  Process  Models 

Short-Term  Memory  Models 
Visual  Scann ing/De tect ion  Models 

GRC  (General  Research  Corporation) 

MARSAM  II  (Multiple  Airborne  Reconnaissance  Sensor 
Assessment  Model) 

VISTRAC  (VISual  Target  Recognition  and  Acquisition) 
CRF.SS/SCREEN  (Combined  Reconnaissance,  Surveillance 
SIGINET/SRI  Countersurveillance  Reconnaissance 
Effectiveness  Evaluation) 

Au tone tics  Model 

Detect 

ASTCAt) 

REA/BAC 

Air  Traffic  Control  Models 

Industrial  Inspection  Models 

At  tent ion/Work load  Models 

Faul t-Diagnosis  Models 

HOS  (Human  Operator  Simulation  Model) 

5.0.  Engineering  Design  Simulation 

5.1.  Human-Machine  Integration  Engineering  Research  Simulation 

General  Purpose  Static  Aircraft  Crew  Station 
General  Purpose  Dynamic  Aircraft  Crew  Station 
General  Purpose  Control  Display 
General  Purpose  Multiperson 

5.2.  Human-Machine  Integration  Engineering  Design  Simulation 

Static  Mockup 

Specific  Dynamic  Control /Display 
Specific  Crew  Station 
Outside  Dynamic  Visual  Scene 
Workplace  Simulator 
Multiman  Workstation 
Command  and  Control  Simulation 
Sensor  Simulation 
Computer  Simulation 

6.0.  Procedures 

6.1,  Systems  Engineering  Analytic  &  Management 
6.1.1.  To  Aid  System  Engineering  Analysis 
6.  1  .  1  .  1  .  Manual 

PERT  (Program  Evaluation  and  Review  Tech¬ 
niques 
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Listing  of  Current  HFE  Technologies  (Continued) 


TLA- 1  (Time  Line  Analysis-1) 

CMP  (Critical  Path  Method) 

Expected  Value  Method 

Functional  Flow  Diagrams 

FDI  (Functional  Description  Inventory) 

Function  Allocation  Tradeoffs 

Task  Analysis 

Decision  Tree  Analysis 

Act  ion/ Informat  ion  Requi rement s 

Time  Lines 

Flow  Process 

OSD  (Operational  Sequence  Diagrams) 

Task  Descriptions 

6. 1.1.2.  Computerized 

SW  (Siegel -Wo If  Model) 

SAINT  (Systems  Analysis  of  Integrated 
Networks  of  Tasks) 

PSM  (Pilot  Simulation  Model) 

CAPA  (Computer  Analysis  of  Personnel 
Act ivi ty ) 

GF.RT  (Graphical  Evaluation  and  Review 
Technology) 

FOVEA  (Field  Of  View  Evaluation  Apparatus) 
WSP  (Workload  Simulation  Program) 

K&M  (Reliability  and  Maintainability 
Model ) 

Sensitivity  Analysis 
2.  To  Extract  From  a  Design 
6 .  1 . 2 .  I .  Manua 1 

CURT  (Coordinated  Human  Resources  Tech¬ 
nology) 

HR/DODT  (Human  Resources/Des i gn  of  Op¬ 
tion  Decision  Trees) 

QQPRI  (Qualitative  and  Quantitative  Per¬ 
sonnel  Requirements  Information) 

ISD  (Instructional  System  Development) 

CDB  (Consolidated  Data  Base) 

Comparability  Analysis 
Technical  Order  Function  Evaluation 
TEPPS  (Technique  for  Establishing  Per¬ 
sonnel  Performance  Standards) 

6. 1.2.2.  Computerized 

LCCIM  (Life  Cycle  Cost  Impact  Model) 

TRAMOD  (Training  Requirements  Analysis 
MODe 1 ) 

PAM  (Personnel  Availability  Model) 

LCOM  (Logistics  Composite  Model) 
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Table  I.  Listing  of  Current  HFE  Technologies  (continued) 


t'ORKl.AP  (Computer  i  zed  RE  lot  iousli ip  I.Av- 
out  Planning) 

6.2.  Detailed  Design  Procedures 

6.2.1.  Manual 

Specification  Compliance  Summary  Sheet 
Link  Analysis 

6.2.2.  Computerized 

HECAD  (Human  Engineering  Computer-Aided  Design) 
CATTS  (Continuous  Assessment  of  Task  Time  Stress) 
TB1.A  (Time-Based  Load  Analysis 
RECEP  (RF.lative  Capacity  Estimating  Process) 

CAFES  (Computer  Aided  Function  Allocation  and 
Evaluation  System) 

DMS  (Data  Management  System) 

FAM  (Function  Allocation  Model) 

WAM  (Workload  Assessment  Model) 

CAD  (Computer-Aided  Crew  Station  Design  Model) 
CGS  (Crew  Station  Geometry  Evaluation  Model) 


The  task  study  first  conducted  a  workshop  in  January  1979  with 
twenty-six  key  human  factors  engineers  from  government  and  industry 
to  identify  technological  gaps  and  problems.  From  this  initial  work¬ 
shop,  a  comprehensive  questionnaire  was  developed  based  on  the  tech¬ 
nology  categories  identified  in  Table  1,  to  further  determine  the 
relative  degree  of  use  of  these  technologies  in  the  Laboratory  and 
Systems  development  environments. 

This  questionnaire,  based  on  the  technology  areas  in  Table  I, 
was  used  to  assess  the  degree  of  utilization  of  the  respective  tech¬ 
nologies  throughout  the  research,  development.  Lest  and  evaluation 
phases  of  the  system  acquisition  process.  The  questionnaire  was  dis¬ 
tributed  to  more  than  twenty  (20)  industrial  organizat ions  with  known 
human  factors  experience.  Responses  covered  thirty-nine  (39)  speci¬ 
fic  weapon  and  subsystem  development  programs  including  fighter  and 
bomber  aircraft  as  well  as  certain  command  and  control  systems  cover¬ 
ing  a  ten  year  period  from  the  late  sixties  to  the  late  seventies. 

A  comparable  questionnaire  was  administered  to  over  one  hundred  (100) 
government  human  factor  engineers  in  Air  Force  Laboratories  or  engi¬ 
neering  development  organizations. 

Table  XI  summarizes  the  findings  of  percentage  use  of  the  six 
technology  categories  tabled  across  industry  managers  and  project  en¬ 
gineers  vs.  government  applications  and  laboratory  responses.  It  is 
evident  from  these  percentages  that  reference  data  sources  receive 
a  high  utilization  rate  across  the  board  whereas  models  have  a  low 
utilization  rate  by  all  respondent  categories.  This  finding  probably 
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Table  II.  Average  Percent 

Use  of 

General  HFE 

Technology 

Ca  tegori es 

INDUSTRY 

GOVERNMENT 

General  HFF.  Technology 

! .  Man¬ 
agers 

2.  Pro¬ 
jects 

3.  Appli¬ 
cations 

4.  Labor¬ 
atories 

(15) 

(39) 

(71) 

(41) 

Reference  Data  Source 

67  % 

557 

46% 

42% 

Experimental 

Human  Performance  Met- 

52Z 

297 

22% 

51% 

rics 

i*n 

302 

28% 

33% 

Mode  1  s 

Engineering  Design  Simu- 

132 

10% 

6% 

17% 

lat ion 

68% 

50% 

34% 

39% 

Procedures 

26% 

21% 

16% 

14% 

reflects  on  the  technological  lag  encountered  in  more  sophist icated 
and  quantitative  mathematical  modeling  methods. 

Table  III  breaks  down  the  broad  category  response  into  selected 
detailed  HFE  technology  areas  by  industry  and  government  respondents. 
These  data  further  illustrate  that  for  all  respondent  categories  that 
the  more  established  technologies  (Reference  Data  Sources,  Statisti¬ 
cal  Research  Methods,  Anthropometric  Measures  and  Design  Simulation 
Methods)  are  the  most  frequently  used.  The  newer  technologies  involv¬ 
ing  mathematical  modeling  and  computerized  techniques  are  not  uti¬ 
lized  much.  This  fact  underscores  the  technological  lag  problem.  At 
the  bottom  of  the  table  the  intercorrelations  between  respondent  cat¬ 
egories,  demonstrate  the  lack  of  relationships  between  laboratory 
scientists'  utilization  of  these  technologies  and  the  applications 
human  factors  engineers  ( Y j  ^  =  .85,  r7  ^  -  .72,  r^  ^  =  .76).  Where¬ 
as  the  correlation  between  t4chnology  us4  between  managers,  project 
personnel  and  government  applications  personnel  are  somewhat  higher. 
One  possible  interpretation  of  these  findings  is  that  the  technology 
transfer  between  the  technology  developers  (laboratory  scientists) 
and  the  human  factors  engineers  (both  management  and  project)  remains 
a  problem  of  some  significance. 

This  technology  transfer  problem  may  stem  from  several  sources, 
but  two  facets  merit  special  concern.  First,  at  least  in  the  USAF 
Research  and  Development  Community,  there  is  no  well  defined  manage¬ 
ment  mechanism  to  insure  efficient  feed-forward  from  the  laboratories 
to  the  developers  nor  is  there  a  feed-back  formalized  procedure  from 
the  operational  environment  to  the  laboratory  and  development  pro¬ 
grams.  Secondly,  most  laboratory  research  is  conducted  with  basic  re¬ 
search  (6.1)  or  exploratory  development  (6.2)  funding  and  very  few 
programs  see  the  "prototype"  development  stage  (6.3).  Hence,  the  meth- 
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Table  111.  Average  Pereent  t'se  ol  detailed  11 FK  Technology  Category 


IVta  i  1  ed 

egory 

HFF  Technology  Cat- 

INDUSTRY 

1.  Man-  2.  Pro- 
a  go  r  s  j  e  c  t  s 

(I5l  (39) 

GOVERNMENT 

3.  Appli-  4.  Labor- 
cations  atories 

(71)  (41) 

1  .0. 

Reference  IKitu 

Sources 

67  ” 

55" 

467 

42!” 

2.1. 

Statist! oa 1  Ke s  ea  re  h 
Methods 

7  o'" 

42" 

347 

777 

3.1. 

An  t  h  r opomc  trie  Me  ass¬ 
ures 

18" 

12” 

77 

197 

3..’. 

Physiological  Meas- 
u  re  s 

80’ 

787 

477 

327 

3.3. 

Environmental  Meas¬ 
ures 

447 

267 

387 

217 

3.4. 

Subjective  Opinions 

587 

45!” 

377 

49% 

3.5. 

Automatic  Recording 
Measures 

717 

4  77 

4  1% 

587 

4.1. 

Hi oman  Models 

137 

107 

O  7 

47 

4 . 2 , 

Performance  Models 

177 

137 

11% 

28% 

4.3. 

Process  Models 

1  17 

87 

4% 

127 

5.1. 

Engineering  Research 

S imul at  ion 

687 

437 

26% 

28% 

5.2. 

Engineering  Design 
Simulation 

687 

537 

37% 

44% 

6.  1  .  1  .  1  . 

Manual  Systems  Engi¬ 
neering  Analysis 

657 

5  3% 

36% 

37% 

6. 1 .  1  .2. 

Computerized  Systems 
Engineering  Anal¬ 
ysis 

77. 

6% 

67 

6% 

6. 1.2.1. 

Manual  Procedures 
from  Design 

97 

10% 

15% 

7% 

6.  1  .2.2. 

Computerized  Proce¬ 
dures  from  Design 

5% 

4% 

6% 

2% 

6.2.1  . 

Manual  Detailed  De¬ 
sign  Aids 

40% 

33% 

19% 

16% 

6.2.2. 

Computerized  Detailed 
Design  Aids 

8% 

4% 

3% 

17 

Correlations 


1,2  = 

.95 

V 

II 

.85 

II 

.92 

r2,4  = 

.  72 

2,3  = 

.93 

r3,4  = 

.76 
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odological  tools  involving  computer-based  modeling  and  simulation 
have  no  formal  R&D  management  structure  to  demonstrate  verification, 
validation  and  utility  for  developmental  design  application. 

Based  on  the  analysis  and  evaluation  of  the  questionnaire  re¬ 
sults  on  current  HFE  technology  several  observations  and  conclusions 
can  be  drawn.  First,  although  the  reference  data  sources  are  used  in 
weapon  system  development  more  than  the  advanced  technologies  of  mod¬ 
els  and  computerized  procedures,  it  appears  that  the  requirement  ex¬ 
ists  for  improving  the  package  of  the  reference  data  sources  in  terms 
of  handbooks,  guides  and  the  like  so  they  are  more  amenable  to  de¬ 
sign  use.  In  fact,  the  development  of  a  computerized  reference  data 
bank  may  be  in  order  which  would  allow  HFF.  data  to  be  responsive  to 
specific  design  issues.  A  better  designed  reference  system  which 
could  feed  the  computer  modeling  and  computer-based  procedural i zed 
systems  appears  to  be  in  order.  A  renewed  effort  should  be  launched 
to  transfer  the  computer-based  technology  into  the  implementing  hands 
of  the  applied  human  factors  engineers  with  more  enphasis  on  valida¬ 
tion  of  these  methods  throughout  the  development  and  test  phases. 

The  need  exists  for  integration  of  computer-modeling  and  engineering 
design  simulation  to  developed  hybrid  techniques  to  exploit  the  re¬ 
spective  powers  of  both  in  a  cost-effective  manner. 


FUTURE  REQUIREMENTS 

To  anticipate  the  directions  of  new  methods  and  predict  future 
projections  of  needed  technology,  the  Air  Force  HFE  study  will  again 
be  used  as  a  data  base  along  with  another  study  sponsored  by  the  Air 
Force  Systems  Command  designed  to  define  computer  technology  short¬ 
falls  to  the  year  2000  (C0MTEC-2000)  (Computer  Technology  Forecast 
and  Weapon  Systems  Impact  Study,  1978).  Only  the  man-machine  inter¬ 
face  technology  part  of  this  comprehensive  study  will  be  referred  to 
in  developing  future  requirements. 

Future  Technology  Assessment  Study 

To  assess  future  technology  projections  twenty  nine  (29)  ex¬ 
perts  from  industry,  government  and  academics  were  asked  to  write 
technology  projections,  both  near-term  (3-5  years)  and  far  term  (5- 
15  years)  in  seventeen  (17)  technology  areas  listed  in  Table  IV. 
(Some  areas  were  covered  by  one  or  more  experts.)  The  17  technology 
areas  were  collapsed  into  nine  (9)  human  factors  information  needs 
identified  in  Table  V  and  analyzed  across  three  (3)  levels  -  human 
centered,  human-machine  centered  or  human-machine-mission  focused. 
Most  experts  (54%)  indicated  a  preponderance  of  future  technology 
needs  in  the  area  of  man-machine-mission  as  compared  to  human  (12%) 
or  human-machine  (34%)  applications.  They  also  indicated  informa¬ 
tion  need  for  system  operation  (20%)  and  design  principles/concepts 
(29%)  with  design  data  base/handbook  (15%)  being  the  third  highest 
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Table  IV.  Representative  areas  of  future  HFE  technology  projections 


*  Advanced  display  engineering  technology 

*  Human  factors  in  maintainability 

*  Systems  research  technology 

*  Human  factors  in  safety  engineering 

*  Engineering  anthropometry 

*  Human/computer  interactions 

*  High  thermal  stress 

*  Training  analysis  and  simulation 

*  Biocybernetics 

*  Operator/crew  workload 

*  Computer  modeling  and  simulation 

*  Design  guides  and  data  bases 

*  Advanced  cockpit  technology 

*  Target  detect ion/acqui s i t ion  models 
it  Decision  making 

*  Human  factors  in  manufacturing  technology 


Table 

*  Manpower  and  logistics  factors  in 
system  development 

weapon 

t  ions 

V.  Summary  of  future  HFF.  technology 

projec 

12% 

34% 

54% 

Systems  Operation 

3% 

6% 

1  1% 

207 

T&E  Procedures 

- 

1% 

5% 

6” 

Training  Development 

- 

3% 

4  7 

17, 

HFE 

Training  Requirements 

1% 

4% 

47 

97 

Information 

Personnel  Selection 

- 

1% 

1% 

27 

Needs 

Personnel  Requirements 

1% 

2% 

2% 

57 

Design  Principles/Concepts 

4% 

10% 

157 

297 

Design  Database/Handbook 

3% 

5% 

17 

157 

MENS 

- 

27 

5% 

17. 

H 

H/M 

H/M/M 

U 

A 

I 

M 

C 

S 

A 

H 

S 

N 

I 

I 

N 

0 

E 

N 

HFE  LEVELS  OF  ANALYSIS 


information  need  category.  It  is  fairly  obvious  that  most  of  the  ad¬ 
vanced  thinkers  in  the  human  factors  discipline  believe  that  the  great¬ 
est  needs  for  future  technology  development  are  being  driven  by  the 
requirement  for  a  human-machine-mission  (H-M-M)  systems  analytic  and 
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simulation  capability.  This  human  factors  system  capability  is  in 
turn  being  driven  by  high  technology  advanced  in  hardware/sof tware 
computer  developments  creating  monumental  complex  information  pro¬ 
cessing  requirements  combined  with  mission  threats  becoming  more 
challenging  and  evasive.  H-M-M  systems  analysis  and  simulation  meth¬ 
od..  must  be  developed  to  treat  human,  equipment  and  mission  parame¬ 
ters  in  equivalent  quantitative  terms  in  order  to  isolate  this  re¬ 
spective  contribution  to  overall  systems  effectiveness. 

Williges  and  Topmiller  (in  press)  conclude  from  the  technology 
assessment  that  -  Advances  in  other  life  sciences,  engineering,  and 
computer  science  along  with  consideration  of  energy  supplies,  future 
threat  environments,  and  future  weapon  systems  appear  to  provide  the 
major  impetus  for  future  threat  environments,  and  future  weapon  sys¬ 
tems  appear  to  provide  the  major  impetus  for  future  HFE  technology 
developments.  Current  DOD  research  plans  and  activities  reveal  that 
these  influences  are  driving  the  developments  of  models,  computer  sim¬ 
ulation,  display  assessment,  mul t i-operator  systems,  advanced  cock¬ 
pit  considerations,  management  technology  and  biocybernetics.  Future 
research  is  needed  in  maintenance  design  and  analysis,  engineering 
anthropology,  safety  system  research,  human/computer  interfaces,  and 
human/machine  environments.  Both  near-term  and  long-term  advances 
are  required  in  each  of  these  areas  in  order  to  provide  the  appropri¬ 
ate  technology  base  for  HFE. 

Williges  and  Topmiller  make  the  following  recommendation  regard¬ 
ing  the  implementation  of  computer-based  HFE  technology:  Various 
computer-based  procedures  for  HFE  are  currently  available  and  near- 
term  projections  suggest  even  more  improvements  and  developments  in 
these  procedures.  But,  by  and  large,  this  technology  is  not  being 
implemented  heavily  in  the  design  process.  Several  reasons,  such  as 
the  unavailability  of  the  procedures,  the  lack  of  knowledge  of  the 
procedures,  and  nonexistence  of  contract  requirements  to  use  them 
have  been  given  for  their  sparse  use.  Before  these  various  techniques 
can  be  completely  evaluated,  they  must  be  integrated. 

Little  effort,  however,  has  been  directed  toward  integrating  all 
of  these  approaches  into  a  truly,  computer-based  HFE  design  methodo¬ 
logy  and  providing  a  testbed  for  application  and  further  development. 
Such  an  integrated  methodology  would  include  using  computer-analytic 
models  and  engineering  design  simulation  could  be  effectively  used 
in  interactive  manner  throughout  conceptual  design,  fly-offs  between 
design  configurations  could  be  accomplished  by  large-scale,  total 
mission  simulation,  computer  modeling  could  suggest  alternative  con¬ 
figurations  for  simulator  evaluation,  and  engineering  design  simula¬ 
tion  could  be  used  as  a  means  of  specifying  training  requirements  and 
follow-on  trainer  design. 


The  Human  Factors  Engineering  Technology  Assessment  Study  was 
conducted  by,  and  participated  in,  by  practicing  HFE  scientists,  prac- 
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titioners  and  managers.  The  survey  and  review  results  certainly  yield 
one  common  theme,  viz.  that  computer  technology  has  driven  much  of 
the  advanced  :.'.FE  technology  developmental  needs.  It  was  somewhat  for¬ 
tuitous  that  about  the  same  time  tiie  Air  Force  was  making  a  self-ap¬ 
praisal  of  HFE ,  it  was  also  launching  a  major  study  effort  to  predict 
future  thrusts  of  computer  technology  development  and  the  impact 
these  developments  will  have  on  future  weapon  system  design  concepts. 
The  approach  used  was  to  identify  and  classify  the  areas  upon  which 
logical  estimates  of  risk/benefits  could  be  made  to  assess  future 
investment  strategies  and  to  also  identify  existing  R&D  programs  which 
would  serve  as  technology  drivers  for  the  identified  areas.  This  study 
is  significant  since  it  was  created  and  conducted  under  the  general 
direction  of  the  computer  science  community  and  not  the  NFE  community, 
although  the  study  directors  recognized  that  the  man-machine-interface 
technology  plays  a  significant  role  in  making  the  predictions  con¬ 
sistent  with  three  primary  objectives 

1.  Forcast  the  advancement  of  computer  and  computer-related  telecom¬ 
munication  technologies. 

2.  Assess  the  potential  impact  of  these  technology  advances  on  the 
capabilities  of  existing  or  future  weapon  systems  through  the  year 
2000. 

3.  Determine  the  policies  and  R&D  initiative  required  to  bring  these 
technology  advances  to  fruition  and  to  incorporate  them  in  future 
weapon  systems  capabilities. 

This  study  was  known  as  COMTEC-2000  for  Computer  Technology  Fore¬ 
cast  and  Weapon  System  Impact  Study  for  the  next  20  years. 


COMTEC-2000  MAW-MACHINE  INTERFACE  TECHNOLOGY  SHORTFALLS 

A  special  panel  was  formed  as  part  of  the  overall  study  effort 
to  identify  and  project  man-machine  interface  technology  shortfalls. 
The  panel  was  chaired  by  Donald  L.  Monk  from  our  Laboratory,  with 
members  representing  Flight  Dynamics  Laboratory,  H uman  Resources  La¬ 
boratory  and  the  Mitre  Corporation.  Fig.  1  shows  the  human  computer 
technology  shortfalls  roadmap  which  the  panel  developed  to  define 
the  drivers  of  the  technology  needs  which  in  turn  determine  the  four 
(4)  man-computer  interface  areas  of:  Information  Exchange,  Human-Com¬ 
puter  Symbiosis,  Standards  and  Guidelines,  and  Design  and  Evaluation 
Methodologies.  These  technology  area  mechanisms  were  evaluated  in 
terms  of  state-of-the-art  to  yield  the  final  products  for  better  and 
more  effective  computer  interfaces  designed  for  man. 

Fig.  2  shows  the  risk/payoff  matrix  for  five  (5)  subareas  of  in¬ 
formation  exchange  and  Fig.  3  expands  these  five  areas  in  a  roadmap 
which  shows  how  current  man-machine  interface  research  in  head/eye 
tracking,  neurophysiological/measurement  (F.EG) ,  pilot  workload  as. 
sessment,  multi-function  keyboard  design,  and  voice  control  techniques 


Fig.  2.  Information  exchange 
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serve  as  the  technology  base  to  feed  the  five  (5)  areas  of  improved 
human-computer  interface  information  exchange.  From  Fig.  2  the  areas 
needing  higher  priority  on  the  basis  of  low  to  medium  risk  and  cost 
vs  medium  to  high  payoff  are  improved  graphic  techniques  and  more 
operator  oriented  mul ti-functional  integrated  systems.  Computer  graph 
ic  techniques  are  becoming  extremely  powerful  and  present  the  poten¬ 
tial  for  manipulating  and  controlling  perceptual  cue  to  enhance  the 
effects  of  size,  distance  and  motion  constancies  for  specific  display 
renditions.  Mul t i-f unc t ion  systems  (displays  and  controls)  must  be 
designed  for  comparability  with  certain  human  cognitive  capabilities 
and  limitations  including  short-term  memory. 

Figs.  4  and  5  show  the  risk-payoff  matrix  and  associated  road¬ 
map  respectively  for  the  area  of  human-computer  symbiosis.  It  would 
appear  that  the  potential  for  developing  imbedding  training  techni¬ 
ques  with  interactive  systems  has  not  been  fully  exploited  by  the  hu¬ 
man  factors  and  computer  science  communities.  Principles  for  automa¬ 
tic  self  correcting  and  operator  prompting  techniques  should  be  de¬ 
veloped  exploiting  current  knowledge  of  adaptive  aiding  techniques 
and  artificial  intelligence  algorithms.  Adaptive  techniques  could  he 
used  to  automatically  sense  high  operator  workload  periods  to  tempo¬ 
rarily  store  information  cues  to  pace  and  synchronize  with  operator 
capaci ty/demand  information. 

One  of  the  major  thrusts  in  new  weapon  system  developments  is 
in  the  command,  control  and  communications  (C^)  area.  The  require¬ 
ment  for  voice  control  techniques  for  systems  is  an  advanced  HFF. 
technological  need  which  emphasizes  the  burgeoning  developments  in 
"human  centered"  design. 

A  requirement  also  exists  for  a  "Trainable  Command  and  Control 
Information  Processing  System".  In  a  modern  tactical  threat  environ¬ 
ment,  it  is  not  simply  the  collection  and  display  of  information  that 
must  be  automated,  but,  of  equal  importance,  the  utilization  of  that 
data.  A  target  nomination  should  be  able  to  trigger  a  sequence  of 
semi-autonomous  processes  sufficient  to  suggest  a  mission  profile  for 
the  approval  of  a  responsible  duty  officer.  It  is  necessary  that  the 
future  tactical  command  and  control  systems  be  modified  on-line  in 
response  to  unforeseeable  demands  of  the  battle  situation. 

Since  Cj  data  base  systems  are  becoming  to  gargantuan,  it  is  nec¬ 
essary  to  develop  "knowledge-based  fusion  systems"  where  fusion  is 
not  only  the  merging  of  multi-sensor  data,  but  the  interpretation  of 
these  data  achieved  by  their  integration  with  other  data  and  know¬ 
ledge  of  a  symbolic  nature.  The  need  for  new  ways  to  display  and  in¬ 
tegrate  these  fusion  systems  present  a  challenging  HFF.  problem. 

Even  given  the  current  embryonic  state  of  human-computer  inter¬ 
face  design  it  would  appear  urgent  to  initiate  national  and  interna¬ 
tional  programs  to  develop  standards  and  guidelines  for  symbology, 
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display  formats,  command  languages  and  equipment  with  Human  Factors 
Engineers  and  Ergonomists  playing  a  central  role  in  their  development 
Figs.  6  and  7  again  outline  the  risk/payoff  analysis  and  the  roadmap 
for  developing  these  standards  and  guidelines.  Certain  triservice 
(Army,  Navy  and  Air  Force)  programs  including  the  Joint  Tactical  In¬ 
formation  Display  System  (JTIDS),  Digital  Avionics  Information  System 
(DAIS)  and  various  Command/Control  programs  designed  for  interopera¬ 
bility  could  presently  use  preliminary  standards  and  guidelines  in 
these  areas.  Eventually,  the  DOD  developed  standards  will  require  in¬ 
ternational  coordination  and  agreement  for  NATO  joint  force  implemen¬ 
tation.  It  is  probably  not  too  early  to  establish  NATO  committees  to 
initiate  preliminary  development  of  international  standards  in  the 
areas . 

Finally,  programs  should  be  established  to  develop  design  and 
evaluate  methodologies.  A  listing  of  eight  (8)  subareas  to  be  addres¬ 
sed  in  such  programs  are  included  in  Fig.  8  with  the  proposed  roadmap 
for  integrating  these  efforts  in  Fig.  9.  Function  and  task  taxonomies 
are  needed  along  with  integrated  modeling  approaches  to  quantitative¬ 
ly  specify  human-computer  interface  requirements  to  anticipate  new- 
conceptual  design  requirements  for  advanced  computer-based  systems 
with  the  ultimate  goal  of  predicting  overall  systems  performance  and 
effectiveness  well  in  advance  of  committing  to  a  particular  design 
configuration. 

The  C0MTECH-2000  Summary  Report  concludes  with  the  following 
statement:  Man-machine  interface  will  improve  in  response  to  the 
commercial  market  competition  for  sales  to  the  layman  of  increasing¬ 
ly  sophisticated  devices.  The  trend  will  be  for  devices  to  be  self¬ 
describing,  assisting,  the  uses  in  their  operation  and  maintenance. 

A  continuing  important,  relatively  unchanging  role  for  men  in  the 
Air  Force  is  forecast;  and  the  national  manpower  pool  from  which  they 
will  be  drawn  will  increasingly  have  computer  experiences  -  it  is  es¬ 
timated  that  by  1985  some  75  percent  of  the  nations' work  force  will 
be  working  with  computers.  Man-machine  dialogue  will  improve  as  bet¬ 
ter  models  of  the  data  base,  the  uses,  and  his  objectives  are  incor¬ 
porated  into  the  computer.  Modalities  of  dialogue  will  range  from 
keyboard,  light-pen,  and  touch,  through  tablet  and  voice,  to  eye-move¬ 
ment  and  electroencephalogram.  The  latter  three  are  of  potentially 
great  interest  to  the  Air  Force  and  may  not  fully  develop  from  com¬ 
mercial  R&D  alone.  A  more  intelligent,  situations-dependent  use  will 
be  made  of  display  space.  Similarily,  programmable  manipulands  (e.g. 
soft-copy  keyboards)  will  be  developed  by  the  commercial  sector.  Fi¬ 
nally,  there  is  a  tantalizing  prospect  of  computer  mediated  transla¬ 
tions  (in  a  limited  context)  from  language  to  language  and/or  from 
verbal  to  pictorial  representaions  (ideographs). 
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SUMMARY  AND  CONCLUSIONS 

This  paper  has  attempted  to  review  some  of  the  historical  de¬ 
velopment,  current  state-of-the  technology  and  practices  as  well  as 
estimates  of  future  technology  directions  and  trends  in  methods, 
techniques  and  data  bases  for  man-machine  interface  design  and  their 
interrelationships  with  system  design  considerations. 

Past,  and  to  some  extent  current  methods  have,  and  are,  using 
primarily  the  established  data  bases  and  manual  analytic  and  design 
tools.  The  more  advanced  technologies  which  employ  computer-based 
analysis  and  simulation  await  full  application  to  the  design  process 
pending  acceptance  and  technical  upgrade  of  the  practicing  human  fac¬ 
tors  engineers  working  in  the  system  development  and  acquisition  pro¬ 
cess.  It  is  anticipated  that  within  the  next  five  (5)  to  ten  (10) 
years  we  will  see  an  upsurge  in  the  exploitation  and  use  of  computer- 
based  simulations  and  modeling,  a  rapid  assimulation  of  engineering 
design  simulation  employing  man-in-the-loop  evaluations  of  advanced 
man-machine  interface  concepts  including  voice  control  and  the  use  of 
adaptive  neurophysiological  control  techniques  to  generate  display 
information  requirements  as  a  function  of  mission  demand.  We  should 
also  see  an  increased  emphasis  on  man-machine  design  considerations 
in  overall  "cost-of-ownership"  and  "life-cycle-costing"  estimations 
earlier  in  the  design  and  development  sequence.  Overall  mission  simu¬ 
lation  capability  will  be  an  increasing  requirement  for  system  de¬ 
velopment  programs.  It  is  also  anticipated  that  greater  use  will  be 
made  of  "hybrid"  simulation  methods  where  the  combined  powers  of  com¬ 
puter-simulation  and  man-in-the-loop  physical  simulation  will  be  used 
to  anticipate  the  increasing  complexities  of  operational  tasks  which 
are  being  driven  by  more  sophisticated  threats  and  equipment. 
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